Display control apparatus

ABSTRACT

An image processing unit identifies the shape of an obstacle that is identified from an area that appears in a peripheral image based on an image captured by a camera. The shape of the obstacle includes at least a tilt of a section of the obstacle in a road-surface direction. The section of the obstacle faces a vehicle. The image processing unit generates a superimposed image in which a mark image that is generated as a pattern that indicates the identified obstacle is superimposed onto a position that corresponds to the obstacle in the peripheral image. At this time, the image processing unit variably changes properties of the mark image based on the tilt of the obstacle identified by an obstacle identifying unit. The image processing unit then displays the generated superimposed image on display apparatus.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a national stage application under 35 U.S.C.071(c) of PCT Application No. PCT/JP2017/016367, filed on Apr. 25, 2017,which is based on and claims the benefit of priority from JapanesePatent Application No. 2016-088183, filed on Apr. 26, 2016, thedescriptions of which are incorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to a display control apparatus thatdisplays an image on a display apparatus that is provided inside avehicle, the image being captured by a camera that is mounted in thevehicle.

BACKGROUND

A technology in which a camera that is mounted in a vehicle captures animage of a predetermined area in the periphery of the vehicle and thecaptured image that has been captured is displayed on a displayapparatus, such as a display, that is provided inside the vehicle isknown.

In such a technology, an obstacle detection display technology in whichinformation indicating an obstacle that is present in the captured imageis displayed together with the captured image is known. For example, PTL1 discloses a technology in which, in addition to an image of a vehiclerear portion being displayed in a display apparatus, a pattern thatindicates the direction and distance of an obstacle detected by a rearsonar is displayed.

CITATION LIST Patent Literature

[PTL 1] JP-A-2000-142284

SUMMARY

In the technology described in PTL 1, a driver can know the approximatedirection of the obstacle and the distance to the obstacle by theposition and color of the pattern that is displayed together with thecaptured image. However, as a result of detailed examination by theinventors, a following issue has been discovered. That is, informationsuch as the position and color of the pattern alone is insufficient forascertainment of specific properties including the shape and orientationof the obstacle.

According to an aspect of the present disclosure, it is desired toprovide a technology for displaying information that enables a driver toeasily ascertain properties including the shape and tilt of an obstaclethat is present in an imaging area in the periphery of a vehicle.

A display control apparatus according to an aspect of the presentdisclosure is configured to display an image in a display apparatus thatis provided inside a vehicle, the image being generated based on animage capturing a predetermined area in a periphery of the vehicle by acamera that is mounted in the vehicle. The display control apparatusincludes an image acquiring unit, an obstacle identifying unit, and acontrol unit. Reference numbers within the parentheses in the claimsindicate corresponding relationships with specific means describedaccording to an embodiment, described hereafter as an aspect, and do notlimit the technical scope of the present disclosure.

The image acquiring unit is configured to acquire a peripheral imagethat is an image based on the image captured by the camera. Theperipheral image is an image that directly shows the image captured bythe camera or an image that is obtained through coordinatetransformation of the image captured by the camera to an image viewedfrom another perspective (such as a bird's-eye-view image). The obstacleidentifying unit is configured to identify a shape of an obstacle thatis identified from an area appearing in the peripheral image acquired bythe image acquiring unit, the shape of the obstacle including at least atilt of a section of the obstacle in a road-surface direction, thesection of the obstacle facing the vehicle.

The control unit is configured to generate a superimposed image in whicha mark image that is generated as a pattern that indicates the obstacleidentified by the obstacle identifying unit is superimposed onto apositon corresponding to the obstacle in the peripheral image, anddisplay the generated superimposed image in the display apparatus.Furthermore, the control unit is configured to variably changeproperties of the mark image based on the tilt of the obstacleidentified by the obstacle identifying unit.

As a result of the display control apparatus configured as describedabove, the properties of the mark image that is the pattern thatindicates an obstacle can be freely changed based on the shape includingat least the tilt of the obstacle. In addition, as a result of a driverof the vehicle viewing the mark image that is superimposed onto theperipheral image, the driver can easily ascertain the shape and tilt ofthe obstacle.

BRIEF DESCRIPTION OF DRAWINGS

The above-described object, other objects, characteristics, andadvantages of the present disclosure will be further clarified throughthe detailed description hereafter, with reference to the accompanyingdrawings. An overview of the drawings is as follows:

FIG. 1 is a block diagram of an overall configuration of an onboarddisplay system;

FIG. 2 is a flowchart of the steps in a distance measurement process;

FIG. 3 is a flowchart of the steps in an obstacle display process;

FIG. 4A is a diagram of a display example of a superimposed image;

FIG. 4B is a diagram of a display example of the superimposed image;

FIG. 4C is a diagram of a display example of the superimposed image;

FIG. 5A is a diagram of transitions in the superimposed image intime-series;

FIG. 5B is a diagram of transitions in the superimposed image intime-series;

FIG. 5C is a diagram of transitions in the superimposed image intime-series;

FIG. 6A is a diagram of a variation of a mark image;

FIG. 6B is a diagram of a variation of the mark image;

FIG. 6C is a diagram of a variation of the mark image;

FIG. 6D is a diagram of a variation of the mark image;

FIG. 6E is a diagram of a variation of the mark image;

FIG. 6F is a diagram of a variation of the mark image;

FIG. 7 is a diagram of a display example of the superimposed image;

FIG. 8 is a diagram of a display example of the superimposed image;

FIG. 9 is a diagram of a display example of the superimposed image;

FIG. 10A is a diagram of a border of a detected obstacle;

FIG. 10B is a diagram of a display example of the superimposed image;

FIG. 11A is a diagram of a display example of the superimposed image ata far distance;

FIG. 11B is a diagram of a display example of the superimposed image ata near distance;

FIG. 11C is a diagram of a display example of the superimposed image ata very near distance;

FIG. 12 is a diagram of a display example of the superimposed image;

FIG. 13 is a diagram of a display example of the superimposed image;

FIG. 14A is a diagram of a display example of the superimposed image;

FIG. 14B is a diagram of a display example of the superimposed image;

FIG. 15A is a diagram of a display example of the superimposed image;

FIG. 15B is a diagram of a display example of the superimposed image;

FIG. 16 is a diagram of a display example of the superimposed image;

FIG. 17 is a diagram of a display example of the superimposed image;

FIG. 18A is a diagram of a display example of the superimposed image ata far distance;

FIG. 18B is a diagram of a display example of the superimposed image ata near distance;

FIG. 18C is a diagram of a display example of the superimposed image ata very near distance;

FIG. 19A is a diagram in which an image of an own vehicle is arranged inan original image including peripheral vehicles;

FIG. 19B is a diagram of the borderlines of the peripheral vehicles thatare actually detected;

FIG. 19C is a diagram in which icons indicating vehicles are arranged tocoincide with detection lines;

FIG. 19D is a diagram in which the icons indicating vehicles aresuperimposed on the original images of the peripheral vehicles;

FIG. 19E is a diagram in which transparent-colored icons indicatingvehicles are superimposed on the original images of the peripheralvehicles; and

FIG. 20 is a diagram in which an icon indicating a vehicle issuperimposed on an original image of a peripheral vehicle.

DESCRIPTION OF EMBODIMENTS

An embodiment of the present disclosure will hereinafter be describedwith reference to the drawings. The present disclosure is not limited tothe embodiment described below and may be carried out according tovarious modes.

[Description of an Onboard Display System Configuration]

A configuration of an onboard display system 10 according to theembodiment will be described with reference to FIG. 1. The onboarddisplay system 10 is a system that is mounted in a vehicle 1 and assistsin driving by displaying an image that is generated based on an imagecaptured by a camera. As shown in an example in FIG. 1, the onboarddisplay system 10 includes an image processing unit 14, a camera 11, adistance measuring unit 12, and a display unit 13. The camera 11 isconnected to the image processing unit 14.

The camera 11 is an imaging apparatus that is set so as to face theperiphery, such as ahead, to the side, or to the rear, of the vehicle 1.The camera 11 is configured to capture an image of a peripheral area ofthe vehicle 1, and output data of an image (also referred to, hereafter,as a captured image) that expresses the image that has been captured tothe image processing unit 14.

The distance measuring unit 12 is a sensor that is configured to acquireinformation by scanning the area imaged by the camera 11. Theinformation indicates the distance between an obstacle (such as anothervehicle, a pedestrian, or a wall or a column of a building) that ispresent in the scanned area and the vehicle 1, and the direction of theobstacle when viewed from the vehicle 1. For example, the distancemeasuring unit 12 is realized by an ultrasonic sonar, a millimeter-waveradar, a laser radar, a stereo camera, a monocular camera, a peripherymonitoring camera, or the like. The position, the shape of a border, thetilt of a face, and an approximate width of the obstacle can berecognized from the measurement results obtained by the distancemeasuring unit 12.

The display unit 13 is a display that is configured to display the imageinformation provided by the image processing unit 14. For example, thedisplay unit 13 is provided in a location that is easily visible to adriver of the vehicle 1, such as in an instrument panel of the vehicle1.

The image processing unit 14 is an information processing apparatus thatis mainly configured by a central processing unit (CPU), a random accessmemory (RAM), a read-only memory (ROM), a semiconductor memory such as aflash memory, an input/output interface, and the like (not shown). Forexample, the image processing unit 14 is realized by a microcontrollerin which functions of a computer system are consolidated. The functionsof the image processing unit 14 are actualized by the CPU running aprogram that is stored in a non-transitory tangible storage medium suchas the ROM or the semiconductor memory. The image processing unit 14 maybe configured by a single or a plurality of microcontrollers. The methodfor actualizing the functions of the image processing unit 14 is notlimited to software. Some or all of the functions may be actualizedthrough use of hardware combining logic circuits, analog circuits, andthe like.

The image processing unit 14 performs a distance measurement process andan obstacle display process based on the above-described program. Adetailed description of these processes will be given hereafter.

[Description of the Distance Measurement Process]

The steps in the distance measurement process performed by the imageprocessing unit 14 will be described with reference to a flowchart inFIG. 2. The distance measurement process is repeatedly performed at apredetermined control cycle during operation of the onboard displaysystem 10.

At step S100, the image processing unit 14 measures the distance to anobstacle that is present in the periphery of the vehicle 1 using thedistance measuring unit 12 and acquires positional information relatedto the obstacle. Specifically, the image processing unit 14 continuouslyscans the periphery of the vehicle 1 using detection waves of a radar, asonar, or the like that configures the distance measuring unit 12, andreceives reflected waves from the obstacle. The image processing unit 14thereby acquires the positional information that indicates adistribution of the distance to an obstacle present in the scanned area.Alternatively, the positional information that indicates a distributionof the distance to an obstacle may be acquired through use of a knownimage recognition technology in which the distance to an object isrecognized based on an image that is captured by a stereo camera, amonocular camera, a periphery monitoring camera, or the like.

At step S102, the image processing unit 14 stores the positionalinformation acquired at step S100, that is, the information thatindicates a distribution of the distance between the vehicle 1 and anobstacle in the memory within the image processing unit 14. After stepS102, the image processing unit 14 returns the process to step S100.

[Description of the Obstacle Display Process]

The steps in the obstacle display process performed by the imageprocessing unit 14 will be described with reference to a flowchart inFIG. 3. The obstacle display process is repeatedly performed at apredetermined control cycle during operation of the onboard displaysystem 10, concurrently with the above-described distance measurementprocess.

At step S200, the image processing unit 14 acquires the latest capturedimage amounting to a single frame from the camera 11. At step S202, theimage processing unit 14 performs a coordinate transformation on thecoordinates of the pixels that configure the captured image acquired atstep S200 using a known technique for bird's-eye-view conversion, andthereby converts the captured image of the camera 11 to abird's-eye-view image that simulates a state of overlooking from aviewpoint set above the vehicle 1.

At step S204, the image processing unit 14 reads the latest positionalinformation acquired through the above-described distance measurementprocess (see FIG. 2) from the memory. At step S206, the image processingunit 14 generates a mark image based on the positional information readat step S204 and the shape of the obstacle appearing in thebird's-eye-view image generated at step S202.

The mark image is a pattern used to indicate the obstacle that ispresent in the bird's-eye-view image. Specific properties of the markimage generated at this time will be described hereafter. At step S208,the image processing unit 14 generates a superimposed image in which themark image generated at step S206 is superimposed onto a position thatcorresponds to the obstacle that appears in the bird's-eye-view imagegenerated at step S202. The image processing unit 14 then displays thegenerated superimposed image in the display unit 13.

Here, the image processing unit 14 is configured to generate thesuperimposed image by changing the properties of the mark image to besuperimposed onto the image of the obstacle, based on the state, such asthe shape, tilt, position, and color, of the obstacle in the capturedimage captured by the camera 11 and identified by the distance measuringunit 12. For example, the properties of the mark image herein includesthe shape, size, tilt, flashing, color, concentration, and transparencyof the pattern. Hereafter, specific application examples of the markimage to be superimposed onto the image of the obstacle will bedescribed with reference to FIG. 4 to FIG. 20.

FIG. 4: Application Example 1

FIG. 4A, FIG. 4B, and FIG. 4C are examples of the superimposed imagedisplayed in the display unit 13. A vehicle image 101 that is a virtualimage showing an external appearance of the vehicle 1 (that is, an ownvehicle) is arranged in the superimposed images. Hereafter, the vehicleimage 101 is presumed to be arranged in a similar manner in thesuperimposed images described in other application examples. The vehicleimage 101 is inserted by the image processing unit 14 when the capturedimage captured by the camera 11 undergoes the bird's-eye-viewconversion. In addition, the mark image that is the pattern thatindicates the obstacle appearing in the bird's-eye-view image isinserted in the superimposed image by the image processing unit 14.

In a case in FIG. 4A, a state in which an image 201 of a narrow,elongated, columnar obstacle appears on a front side of the vehicleimage 101 is assumed. In the case in FIG. 4A, the image processing unit14 generates a mark image 301 that is composed of a plate-shaped patternthat has a width similar to the width of the obstacle image 201. Theimage processing unit 14 then generates the superimposed image bysuperimposing the mark image 301 to coincide with the position of theobstacle image 201 in the bird's-eye-view image.

In a case in FIG. 4B, a state in which an image 202 of a wall-likeobstacle that has a width similar to that of the vehicle 1 appears onthe front side of the vehicle image 101 is assumed. In the case in FIG.4B, the image processing unit 14 generates a mark image 302 that iscomposed of a plate-shaped pattern that has a width that is wider thanthat in the case in FIG. 4A, based on the width of the obstacleappearing in the bird's-eye-view image. The image processing unit 14then generates the superimposed image by superimposing the mark image302 to coincide with the position of the obstacle image 202 in thebird's-eye-view image.

In a case in FIG. 4C, a state in which an image 203 of a wall-likeobstacle that opposes the vehicle image 101 in a state in which a faceis tilted in relation to a front direction of the vehicle image 101appears diagonally to the front of the vehicle image 101 is assumed. Inthe case in FIG. 4C, the image processing unit 14 generates a mark image303 that is composed of a pattern in which a plate-shaped face is tiltedto match the tilt of the obstacle image 203 in relation to the frontdirection of the vehicle image 101. The image processing unit 14 thengenerates the superimposed image by superimposing the mark image 303 tocoincide with the position of the obstacle image 203 in thebird's-eye-view image.

In the cases in FIG. 4A, FIG. 4B, and FIG. 4C, the image processing unit14 is configured to be capable of changing the orientation of the markimage composed of the plate-shaped pattern, based on the tilt in aroad-surface direction of the section at which each of the obstacles201, 202, and 203 faces the vehicle 1. Specifically, the imageprocessing unit 14 recognizes the tilt in the road-surface direction ofthe section at which each of the obstacles 201, 202, and 203 faces thevehicle 1 based on the shape of the border of each of the obstacles 201,202, and 203 indicated by the positional information acquired by thedistance measuring unit 12. The image processing unit 14 then generatesthe mark images 301, 302, and 303 that are each composed of a geometricpattern that has the same tilt as the tilt of the respective obstacles201, 202, and 203 on the coordinate system of the bird's-eye-view image.The image processing unit 14 superimposes the mark images onto theimages of the obstacles 201, 202, and 203.

FIG. 5: Application Example 2

The image processing unit 14 may be configured to periodically flash themark image that is displayed so as to overlap the obstacle image. Inaddition, as shown in examples in FIG. 5A, FIG. 5B, and FIG. 5C, thelength of a cycle at which a mark image 304 is flashed may be changedbased on the length of the distance between the vehicle 1 and anobstacle 204. As a result, the driver can accurately ascertain thedistance to the obstacle.

A case in FIG. 5A is a time series showing the transitions over time inthe superimposed image that is displayed by the image processing unit 14in a state in which the distance between the vehicle 1 and the obstacleis relatively far. In the case in FIG. 5A, the image processing unit 14displays the superimposed image in which the mark image 304 that issuperimposed onto the obstacle image 204 is flashed at a relatively longcycle.

A case in FIG. 5B is a time series showing the transitions over time inthe superimposed image that is displayed by the image processing unit 14in a state in which the distance between the vehicle 1 and the obstacleis relatively near (that is, nearer than that in the case in FIG. 5A).In the case in FIG. 5B, the image processing unit 14 displays thesuperimposed image in which the mark image 304 that is superimposed ontothe obstacle image 204 is flashed at a cycle that is relatively shorterthan that in the case in FIG. 5A.

A case in FIG. 5C is a time series showing the transitions over time inthe superimposed image that is displayed by the image processing unit 14in a state in which the distance between the vehicle 1 and the obstacleis very near (that is, even nearer than that in the case in FIG. 5B). Inthe case in FIG. 5C, the image processing unit 14 displays thesuperimposed image in which the mark image 304 that is superimposed ontothe obstacle image 204 is continuously displayed or flashed at a cyclethat is even shorter than that in the case in FIG. 5B.

FIG. 6: Application Example 3

As shown in examples in FIG. 6A, FIG. 6B, FIG. 6C, FIG. 6D, FIG. 6E, andFIG. 6F, the image processing unit 14 may be configured to superimpose amark image that may include a figure comes in a variety of shapes, asymbol, an icon, or character information onto an image of an obstacle.

A case in FIG. 6A is an example of a superimposed image in which a markimage 305 that is composed of a plate-shaped pattern is superimposedalong the face of an obstacle image 205. A case in FIG. 6B is an exampleof a superimposed image in which a mark image 306 that is composed of apolygonal-line-shaped pattern is superimposed along the face of theobstacle image 205. A case in FIG. 6C is an example of a superimposedimage in which a mark image 307 that is composed of a wavy-line-shapedpattern is superimposed along the face of the obstacle image 205.

A case in FIG. 6D is an example of a superimposed image in which a markimage 308 that is composed of a dot-sequence-shaped pattern issuperimposed along the face of the obstacle image. A case in FIG. 6E isan example of a superimposed image in which a mark image 309 that iscomposed of an icon that includes a symbol that prompts caution issuperimposed along the face of the obstacle image 205. A case in FIG. 6Fis an example of a superimposed image in which a mark image 310 that iscomposed of character information that indicates the distance to theobstacle is superimposed along the face of the obstacle image 205.

FIG. 7: Application Example 4

The image processing unit 14 may be configured to arrange a mark imagefor an obstacle that corresponds to a course on which the vehicle 1 ispredicted to advance or an area obtained by the vehicle width beingextended in frontward and rearward directions of the vehicle length. Inthis case, the mark image may not be displayed in other areas even whenan obstacle is detected. Specifically, the image processing unit 14predicts the course of the vehicle 1 by acquiring vehicle informationthat indicates a steering state of the vehicle 1 and the like. The imageprocessing unit 14 then identifies the area of the predicted course orthe area in the frontward and rearward directions of the vehicle lengthin the bird's-eye-view image based on information, such as the vehiclewidth and the vehicle length of the vehicle 1, registered in advance.

A case in FIG. 7 is an example of a superimposed image that is displayedin a state in which a course on which the vehicle 1 turns in a leftwarddirection is predicted. In the case in FIG. 7, in an obstacle image 206,a mark image 311 is arranged along an area in a lateral direction thatcorresponds to an area of the locus of the predicted course on which thevehicle 1 turns in the leftward direction and an area obtained by thevehicle width being projected in the frontward direction. As a result,an obstacle in an area with which the vehicle 1 is likely to come intocontact can be presented in a manner that is easily understandable tothe driver.

FIG. 8: Application Example 5

The image processing unit 14 may be configured to arrange a mark imagefor an obstacle that corresponds to an area with reference to thevehicle width of the vehicle 1, taking into consideration the likelihoodof contact between the vehicle 1 and the obstacle. In this case, asshown in an example in FIG. 8, the configuration may be such that themark image is arranged over an area that is slightly wider than thevehicle width of the vehicle 1.

The case in FIG. 8 is an example of a superimposed image that isdisplayed in a state in which an obstacle is present diagonally to thefront of the vehicle 1. In the case in FIG. 8, in an obstacle image 207,a mark image 312 is arranged in a section that corresponds to a displayarea that is obtained by an area obtained through projection of thevehicle width of the vehicle 1 in the frontward direction being enlargedby a predetermined distance to the left and right. As a result, anobstacle in an area with which the vehicle 1 is likely to come intocontact can be presented in a manner that is easily understandable tothe driver.

FIG. 9: Application Example 6

The image processing unit 14 may be configured to arrange a mark imagerelated to the obstacle in an area that is wider than a width of aborderline (also referred to, hereafter, as a detection line) thatindicates a shape of an obstacle that is detected by the radar or thesonar of the distance measuring unit 12. Specifically, the imageprocessing unit 14 identifies the width of the detection line of theobstacle that is indicated by the positional information acquired by thedistance measuring unit 12. The image processing unit 14 then identifiesthe area over which the mark image is arranged with reference to thewidth of the detection line.

A case in FIG. 9 is an example of a superimposed image that is displayedin a state in which an obstacle is present in front of the vehicle. Inthe case in FIG. 9, a mark image 313 is arranged in a section of anobstacle image 208 that corresponds to a display area that is obtainedby the width of the detection line that has actually been detected beingextended by a predetermined distance to the left and right. As a result,the scale of the obstacle can be accurately presented to the driver evenin cases in which the width of the actual obstacle is greater than thewidth of the detection line.

FIG. 10: Application Example 7

The image processing unit 14 may be configured to recognize a border ofan obstacle, which has been detected by the radar or the sonar of thedistance measuring unit 12, from the bird's-eye-view image using imagerecognition. The image processing unit 14 may then arrange a mark imagerelated to the obstacle along the recognized border.

Specifically, as shown in an example in FIG. 10A, the image processingunit 14 extracts a feature quantity, such as an edge, from an image areasurrounding the detection line in an obstacle image 209 appearing in thebird's-eye-view image. The image processing unit 14 then detects aborder of the obstacle in a vertical direction based on the extractedfeature quantities. In addition, as shown in an example in FIG. 10B, theimage processing unit 14 superimposes a mark image 314 that has a widthalong the borders of the obstacle image 209 onto the obstacle image 209.As a result, the scale of the obstacle can be accurately presented tothe driver even in cases in which the width of the actual obstacle isgreater than the width of the detection line.

FIG. 11: Application Example 8

The image processing unit 14 may be configured to change the properties(such as the shape, size, color, and transmittance) of the mark image tobe superimposed onto an image of an obstacle based on thefarness/nearness of the distance between the vehicle 1 and the obstacle.As a result, the driver can accurately ascertain the distance to theobstacle.

A case in FIG. 11A is an example of a superimposed image that isdisplayed by the image processing unit 14 in a state in which thedistance between the vehicle 1 and the obstacle is relatively far. Inthe case in FIG. 11A, the image processing unit 14 displays asuperimposed image in which a mark image 315 that has a relatively smallshape is superimposed onto an obstacle image 210.

A case in FIG. 11B is an example of a superimposed image that isdisplayed by the image processing unit 14 in a state in which thedistance between the vehicle 1 and the obstacle is relatively near (thatis, nearer than that in the case in FIG. 11A). In the case in FIG. 11B,the image processing unit 14 displays a superimposed image in which amark image 316 that has a shape that is relatively larger than that inthe case in FIG. 11A is superimposed onto the obstacle image 210. Atthis time, the image processing unit 14 may change the color andtransmittance of the mark image 316 to a more emphatic mode than thecolor and transmittance of the mark image 315 in FIG. 11A.

A case in FIG. 11C is an example of a superimposed image that isdisplayed by the image processing unit 14 in a state in which thedistance between the vehicle 1 and the obstacle is very near (that is,even nearer than that in the case in FIG. 11B). In the case in FIG. 11C,the image processing unit 14 displays a superimposed image in which amark image 317 that has a shape that is even larger than that in thecase in FIG. 11B is superimposed onto the obstacle image 210. At thistime, the image processing unit 14 may change the color andtransmittance of the mark image 317 to a more emphatic mode than thecolor and transmittance of the mark image 316 in FIG. 11B.

FIG. 12: Application Example 9

The image processing unit 14 may be configured to draw the mark image tobe superimposed onto an image of an obstacle so as to be extended to anouter edge of a display area of the superimposed image in a directioncorresponding to an upper side of the obstacle. Specifically, as shownin an example in FIG. 12, the image processing unit 14 draws a markimage 318, which is arranged on an obstacle image 211, so as to beextended to an upper end portion of the display area of the superimposedimage in a direction corresponding to an upper side of the obstacleimage 211. As a result, the mark image related to an obstacle can bepresented to the driver in a mode that emphasizes the height of theobstacle.

FIG. 13: Application Example 10

The image processing unit 14 may be configured to extend the shape ofthe mark image to be superimposed onto an image of an obstacle in thebird's-eye-view image in a radiating manner, taking into considerationdistortion (such as the image being extended in a radiating manner asthe image becomes farther from the center) in the image that occurs whenthe captured image captured by the camera 11 is converted to thebird's-eye-view image. Specifically, as shown in an example in FIG. 13,the image processing unit 14 draws a mark image 319 to be superimposedonto an obstacle image 211 in a shape that is extended in a radiatingmanner with an image position in the bird's-eye-view image correspondingto the position of the camera 11 as the center. As a result, discomfortexperienced by the driver can be reduced.

FIG. 14: Application Example 11

The image processing unit 14 may be configured to draw the mark image tobe superimposed onto an image of an obstacle in a mode in which a lowerend side of the obstacle is emphasized. Specifically, as shown inexamples in FIG. 14A and FIG. 14B, the image processing unit 14emphasizes the lower end side of an obstacle image 213 by drawing eachof mark images 320 and 321 so as to change the color and concentrationof the respective patterns composing the mark images 320 and 321continuously or in steps from the upper end side to the lower end sideof the obstacle image 213. As a result, the mark image can be displayedin a mode in which the section on the lower side of the obstacle that islikely to come into contact with the vehicle 1 is emphasized.

FIG. 15: Application Example 12

The image processing unit 14 may be configured to recognize the color ofthe obstacle from the captured image and draw the mark image using acolor that corresponds to a complementary color of the recognized colorof the obstacle. Specifically, as shown in examples in FIG. 15A and FIG.15B, mark images 322 and 323 that are each composed of a pattern thathas a relationship in which the color is complementary to the color ofthe respective obstacle image 214 or 215 are superimposed onto theobstacle images 214 and 215. As a result, the obstacle image and themark image can be made to complement each other, and visibility by thedriver can be improved.

FIG. 16: Application Example 13

When the obstacle detected by the distance measuring unit 12 is presumedto be a sloped surface, the image processing unit 14 may be configuredto also arrange the mark image in an area further towards the vehicle 1than the detection line that indicates the border of the detectedobstacle.

For example, when the radar or the sonar of the distance measuring unit12 detects a sloped surface, such as an upward slope, an undetectedsloped surface is likely to be continuing towards the vehicle 1 in anarea below a lower limit of the detection area of the radar or the sonarin the vertical direction. Therefore, as shown in an example in FIG. 16,the image processing unit 14 arranges a mark image 324 in an area of animage 216 of a sloped surface that is detected as the obstacle, the areaincluding an area further towards the vehicle image 101 than the actualdetection line. As a result, the attention of the driver can be drawn tothe section of the sloped surface that is not detected by the radar orthe sonar of the distance measuring unit 12.

FIG. 17: Application Example 14

The image processing unit 14 may be configured to draw the mark image tobe superimposed onto an image of an obstacle in a mode in which an areathat is actually detected by the distance measuring unit 12 is givenmore emphasis than other areas. Specifically, as shown in an example inFIG. 17, the image processing unit 14 displays a mark image 325A that iscomposed of a plate-shaped pattern on an obstacle image 217, anddisplays an emphasized image 325B in an image area corresponding to thearea that is detected by the radar or the sonar of the distancemeasuring unit 12. The emphasized image 325B is assumed to be composedof a pattern that is drawn using a more emphatic color, concentration,or transmittance than that of the mark image 325A. As a result, theattention of the driver can be drawn to the border of the obstacle.

FIG. 18: Application Example 15

The image processing unit 14 may be configured to display lines(referred to, hereafter, as grid lines) in the form of squares thatserve as an indicator of the distance between the vehicle 1 and theobstacle in the superimposed image, based on the farness/nearness of thedistance between the vehicle 1 and the obstacle. In addition, the sizeof the squares formed by the grid lines may be variable, based on thefarness/nearness of the distance between the vehicle 1 and the obstacle.As a result, the driver can accurately ascertain the distance to theobstacle.

A case in FIG. 18A is an example of a superimposed image that isdisplayed by the image processing unit 14 in a state in which thedistance between the vehicle 1 and the obstacle is relatively far. Inthe case in FIG. 18A, the image processing unit 14 displays asuperimposed image in which a mark image 326 is superimposed onto anobstacle image 218. However, at this distance, the grid lines that serveas the indicator of the distance between the vehicle 1 and the obstacleare not displayed.

A case in FIG. 18B is an example of a superimposed image that isdisplayed by the image processing unit 14 in a state in which thedistance between the vehicle 1 and the obstacle is relatively near (thatis, nearer than that in the case in FIG. 18A). In the case in FIG. 18B,the image processing unit 14 displays a superimposed image in which amark image 327 is superimposed onto the obstacle image 218, and gridlines composed of relatively large squares are drawn in the periphery ofthe vehicle image 101 and the obstacle image 218.

A case in FIG. 18C is an example of a superimposed image that isdisplayed by the image processing unit 14 in a state in which thedistance between the vehicle 1 and the obstacle is very near (that is,even nearer than that in the case in FIG. 18B). In the case in FIG. 18C,the image processing unit 14 displays a superimposed image in which amark image 328 is superimposed onto the obstacle image 218, and gridlines composed of squares that are smaller than that in the case in FIG.18B are drawn in the periphery of the vehicle image 101 and the obstacleimage 218.

FIG. 19: Application Example 16

When the obstacle detected by the distance measuring unit 12 is presumedto be a vehicle, the image processing unit 14 may be configured todisplay, in a superimposing manner, a mark image that is composed of anicon that represents a vehicle so as to match the orientation and sizeof the vehicle detected as the obstacle.

A case in FIG. 19A shows a bird's-eye-view image generated from thecaptured image captured by the camera 11 in a state in which images 401and 402 of two other vehicles appear in the periphery of the vehicleimage 101 that corresponds to the vehicle 1. FIG. 19B shows thedetection lines that indicate the borders of the other vehicles actuallydetected by the distance measuring unit 12 in the state given as anexample in FIG. 19A.

As shown in an example in FIG. 19C, the image processing unit 14generates mark images 501 and 502 that are each composed of the iconthat represents a vehicle, along the shape and tilt of the detectionlines. The mark images 501 and 502 generated in the case in FIG. 19C arepresumed to be drawn using a non-transparent color. Then, as shown in anexample in FIG. 19D, the image processing unit 14 displays asuperimposed image in which the generated mark images 501 and 502 arerespectively superimposed onto the images 401 and 402 of the othervehicles.

Alternatively, as shown in an example in FIG. 19E, the image processingunit 14 may be configured to superimpose mark images 601 and 602 ontothe images 401 and 402 of the other vehicles. In the mark images 601 and602, the pattern of the vehicle is indicated only by an outline and theinterior is transparent.

FIG. 20: Application Example 17

When drawing a mark image that is composed of an icon that represents avehicle, the image processing unit 14 may be configured to use a singlerepresentative color that is acquired from an original image of anothervehicle onto which the mark image is to be superimposed. Specifically,as shown in an example in FIG. 20, the image processing unit 14 extractsa specific color from an image 403 of another vehicle that appears inthe bird's-eye-view image and displays a mark image 503 that is drawnusing the extracted color so as to overlap the image 403 of the othervehicle. As a result, even in cases in which a complex pattern is drawnon another vehicle that is detected as the obstacle or cases in which acomplex pattern is formed as a result of light passing through thegratings in a parking structure or the like, the mark image can bedisplayed in a simple and easy-to-see mode.

[Effects]

The following effects are achieved by the onboard display systemaccording to the embodiment.

Based on the shape, such as the tilt and size, of an obstacle that isdetected in the periphery of the vehicle 1, the properties, such as theorientation and shape, of the mark image that is the pattern indicatingthe obstacle can be freely changed. In addition, a display mode, such asthe size, color, and flashing, of the mark image can be freely changedbased on the distance to the obstacle. As a result of the driver of thevehicle viewing the superimposed image in which the mark image issuperimposed onto the image of the obstacle in this way, the driver caneasily ascertain the state of the obstacle.

Correspondence to the Configuration According to the Embodiment

The image processing unit 14 corresponds to an example of a displaycontrol apparatus. The processes at steps S200 and S202 performed by theimage processing unit 14 corresponds to an example of a process as animage acquiring unit. The process at step S206 performed by the imageprocessing unit 14 corresponds to an example of a process as an obstacleidentifying unit and a control unit.

Variation Example

A function provided by a single constituent element according to theabove-described embodiments may be divided among a plurality ofconstituent elements. Functions provided by a plurality of constituentelements may be provided by a single constituent element. In addition, apart of a configuration according to the above-described embodiments maybe omitted. Furthermore, at least a pail of a configuration according toan above-described embodiment may be added to or replace a configurationaccording to another of the above-described embodiments. Any modeincluded in the technical concept specified by the wordings of theclaims is an embodiment of the present disclosure.

For example, in the application example 1 (see FIG. 4) described above,a case in which the shape, such as the tilt and width, of the mark imageto be superimposed onto the image of an obstacle is made variable basedon the orientation and width of the obstacle is described. Furthermore,the configuration may be such that the properties, such as the color andpattern, of the mark image are changed based on the tilt of theobstacle.

In addition, according to the above-described embodiment, a case inwhich, upon conversion of a captured image captured by the camera 11into a bird's-eye-view image, the mark image is superimposed onto theimage of an obstacle in the converted bird's-eye-view image isdescribed. In addition, the bird's-eye-view image, the mark image may besuperimposed on the captured image captured by the camera 11 itself.Alternatively, the mark image may be superimposed onto an image obtainedby conversion to an image of a perspective other than thebird's-eye-view image.

The present disclosure can also be actualized in various modes, such asa program for enabling a computer to function as the above-describedimage processing unit 14, and a non-transitory tangible recording mediumsuch as a semiconductor memory, in which the program is recorded.

The invention claimed is:
 1. A display control apparatus that displaysan image on a display apparatus that is provided inside a vehicle, theimage being generated based on an image capturing a predetermined areain a periphery of the vehicle by a camera that is mounted in thevehicle, the display control apparatus comprising: an image acquiringunit that is configured to acquire a peripheral image that is the imagebased on the image captured by the camera; an obstacle identifying unitthat is configured to identify a tilt of an obstacle that is identifiedfrom an area appearing in the peripheral image acquired by the imageacquiring unit, the tilt of the obstacle being a tilt of the obstaclerelative to the vehicle in a road-surface direction; and a control unitthat is configured to generate a superimposed image in which a markimage that is generated as a pattern indicating the obstacle identifiedby the obstacle identifying unit is superimposed onto a positioncorresponding to the obstacle in the peripheral image, and display thegenerated superimposed image on the display apparatus, the control unitbeing configured to variably change properties of the mark image basedon the identified tilt of the obstacle relative to the vehicle in theroad-surface direction as identified by the obstacle identifying unit.2. A display control apparatus that displays an image on a displayapparatus that is provided inside a vehicle, the image being generatedbased on an image capturing a predetermined area in a periphery of thevehicle by a camera that is mounted in the vehicle, the display controlapparatus comprising: an image acquiring unit that is configured toacquire a peripheral image that is the image based on the image capturedby the camera; an obstacle identifying unit that is configured toidentify a tilt of an obstacle that is identified from an area appearingin the peripheral image acquired by the image acquiring unit, the tiltof the obstacle being a tilt of the obstacle relative to the vehicle ina road-surface direction; and a control unit that is configured togenerate a superimposed image in which a mark image that is generated asa pattern indicating the obstacle identified by the obstacle identifyingunit is superimposed onto a position corresponding to the obstacle inthe peripheral image, and display the generated superimposed image onthe display apparatus, the control unit being configured to variablychange properties of the mark image based on a condition of theidentified tilt of the obstacle as identified by the obstacleidentifying unit.
 3. The display control apparatus according to claim 2,wherein: the control unit is configured to display the mark image in atilted mode to match the tilt of the obstacle identified by the obstacleidentifying unit.
 4. The display control apparatus according to claim 2,wherein: the control unit is configured to variably change a displaymode of the mark image to be superimposed onto the obstacle, based on adistance measured by a distance measuring unit that is configured tomeasure a distance between the obstacle and the vehicle.
 5. The displaycontrol apparatus according to claim 2, wherein: the control unit isconfigured to arrange the mark image on an image of an obstacle that ispresent in an area on a locus on which the vehicle is predicted toadvance and an area obtained by a vehicle width of the vehicle beingprojected ahead of or behind the vehicle, among the obstacles identifiedby the obstacle identifying unit.
 6. The display control apparatusaccording to claim 2, wherein: the control unit is configured to arrangethe mark image on an image of an obstacle that is present in an areathat is obtained by an area obtained by a vehicle width of the vehiclebeing expanded by a fixed amount being projected ahead of or behind thevehicle, among the obstacles identified by the obstacle identifyingunit.
 7. The display control apparatus according to claim 2, wherein:the control unit is configured to arrange the mark image in an areaobtained by a border of the obstacle identified by the obstacleidentifying unit being expanded.
 8. The display control apparatusaccording to claim 2, wherein: the obstacle identifying unit isconfigured to identify a border of the obstacle in the peripheral imagebased on a feature quantity in the peripheral image with reference to animage portion of the peripheral image that corresponds to an area inwhich an obstacle is detected by an object sensor that detects anobstacle that is present in the periphery of the vehicle; and thecontrol unit is configured to arrange the mark image along the border ofthe obstacle identified by the obstacle identifying unit.
 9. The displaycontrol apparatus according to claim 2, wherein: the control unit isconfigured to draw the mark image such that the mark image reaches anouter edge portion of a display area of the superimposed image in adirection corresponding to an upper side of the obstacle.
 10. Thedisplay control apparatus according to claim 2, wherein: the imageacquiring unit is configured to acquire, as the peripheral image, abird's-eye-view image in which the image captured by the camera ischanged to an image that is expressed by a bird's-eye-view; and thecontrol unit is configured to display the mark image so as to besuperimposed onto the bird's-eye-view image and draw the mark image soas to have a shape that is extended in a radiating manner with aposition of the camera as a center.
 11. The display control apparatusaccording to claim 2, wherein: the control unit is configured to drawthe mark image in a mode in which a lower end side of the obstacle isgiven more emphasis than an upper end side.
 12. The display controlapparatus according to claim 2, wherein: the control unit is configuredto: draw the mark image using a reference display color when thereference display color does not correspond to a color the same as acolor of the obstacle; and draw the mark image using a display colorthat differs from the reference display color, when the referencedisplay color corresponds to a color the same as the color of theobstacle.
 13. The display control apparatus according to claim 2,wherein: the control unit is configured to arrange the mark image so asto extend to an area that is closer to the vehicle than an area in whichthe obstacle is detected, when the obstacle corresponds to a slopedsurface.
 14. The display control apparatus according to claim 2,wherein: the control unit is configured to draw the mark image in a modein which an image portion of the peripheral image that corresponds tothe area in which the obstacle is detected by an object sensor thatdetects an obstacle is displayed more emphasized than another imageportion that corresponds to the obstacle.
 15. The display controlapparatus according to claim 2, wherein: the obstacle identifying unitis configured to recognize an other vehicle as the obstacle; and thecontrol unit is configured to arrange the mark image, which is a patternthat indicates a vehicle, in a position in the peripheral imagecorresponding to the other vehicle recognized by the obstacleidentifying unit so as to match a tilt of the other vehicle.
 16. Thedisplay control apparatus according to claim 15, wherein: the controlunit is configured to use, as the mark image, a pattern of a vehiclethat is partly drawn using a transparent color or a pattern of a vehiclethat is drawn using a color of the other vehicle recognized from theperipheral image.
 17. A display control apparatus that displays an imageon a display apparatus that is provided inside a vehicle, the imagebeing generated based on an image capturing a predetermined area in aperiphery of the vehicle by a camera that is mounted in the vehicle, thedisplay control apparatus comprising: a processor; a non-transitorycomputer-readable storage medium; and a set of computer-executableinstructions stored in the computer-readable storage medium that causethe processor to implement: acquiring a peripheral image that is theimage based on the image captured by the camera; identifying a tilt ofan obstacle that is identified from an area appearing in the acquiredperipheral image, the tilt of the obstacle being a tilt of the obstaclerelative to the vehicle in a road-surface direction; generating asuperimposed image in which a mark image that is generated as a patternindicating the identified obstacle is superimposed onto a positioncorresponding to the obstacle in the peripheral image, and displayingthe generated superimposed image on the display apparatus; and variablychanging properties of the mark image based on the identified tilt ofthe obstacle relative to the vehicle in the road-surface direction. 18.A display control apparatus that displays an image in a displayapparatus that is provided inside a vehicle, the image being generatedbased on an image capturing a predetermined area in a periphery of thevehicle by a camera that is mounted in the vehicle, the display controlapparatus comprising: a processor; a non-transitory computer-readablestorage medium; and a set of computer-executable instructions stored inthe computer-readable storage medium that cause the processor toimplement: acquiring a peripheral image that is the image based on theimage captured by the camera; identifying a tilt of an obstacle that isidentified from an area appearing in the acquired peripheral image, thetilt of the obstacle being a tilt of the obstacle relative to thevehicle in a road-surface direction; generating a superimposed image inwhich a mark image that is generated as a pattern indicating theidentified obstacle is superimposed onto a position corresponding to theobstacle in the peripheral image, and displaying the generatedsuperimposed image on the display apparatus; and variably changingproperties of the mark image based on a condition of the identified tiltof the obstacle.